Skip to content

Feat/implement phase e#4

Merged
nixrajput merged 14 commits into
mainfrom
feat/implement-phase-e
Jun 21, 2026
Merged

Feat/implement phase e#4
nixrajput merged 14 commits into
mainfrom
feat/implement-phase-e

Conversation

@nixrajput

@nixrajput nixrajput commented Jun 21, 2026

Copy link
Copy Markdown
Owner

Phase E — MySQL + MariaDB drivers

Adds MySQL and MariaDB as siphon drivers, both built on a shared _mysqlcommon package and integrated with the Phase D RunDriverSuite harness. This is the payoff for the Phase D investment: each new engine is a ~30-line thin wrapper.

What's included

  • internal/driver/_mysqlcommon/ — shared code for the two forks:
    • DSN builder, mysqldump/mariadb-dump arg builder, fork detection (DetectFork).
    • A shared Conn implementing the full driver.Conn contract once: Inspect (via information_schema), Backup (shell out to the dump tool, stream stdout), Restore (pipe SQL into the client over stdin), sha256 Verify, and a bounded connect-probe retry (jobs.Retry, mirroring Postgres). Password passed via MYSQL_PWD (no argv leak); stderr discarded (avoids the StderrPipe/cmd.Wait() race the Postgres driver documents).
  • internal/driver/mysql/ and internal/driver/mariadb/ — thin wrappers (~30 lines each) that inject the fork-specific binary names and declare capabilities honestly (Parallel: false — the dump tools are single-threaded). Each registers itself via init().
  • Registration — side-effect imports added to internal/app/drivers.go (not cli/root.go, per the cli-may-not-import-domain depguard boundary).
  • Integration suitesTestSuite_MySQL (mysql:8.0) and TestSuite_MariaDB (mariadb:11) run on the shared RunDriverSuite harness via testcontainers, giving each engine the four contract tests for free (connect/inspect, backup→restore round-trip, cancel propagation, bad-credentials sentinel).
  • CI — installs Postgres/MySQL/MariaDB client tools and runs the full integration suite on the Ubuntu runner (where Docker is available); unit tests continue on Linux/macOS/Windows.
  • Docs — README (status table, requirements, capabilities), CHANGELOG, and DRIVERS.md updated; the MySQL/MariaDB pair is documented as a second worked example of the shared-helper pattern.

Notable decisions / deviations from the original plan

  • Shared Conn (vs the plan's per-driver Inspect/Verify copies) — keeps the fork difference to just the binary names and avoids divergence.
  • Side-effect imports in app/drivers.go, not cli/root.go — the plan's location would have violated depguard.
  • int(port.Num()) for testcontainers v0.42.0 (the MappedPort return type exposes .Num(), not .Int()).
  • Test-target fix: go test ./... silently skips underscore-prefixed dirs, so the _mysqlcommon unit tests were excluded from make test and CI. The Makefile and CI now name ./internal/driver/_*/ explicitly so those tests actually run.

Verification

  • make tidy clean · make test (14 packages, incl _mysqlcommon) · make lint 0 issues · make build ok.
  • go build -tags integration ./... compiles all three driver suites; integration-tagged lint clean.
  • Smoke (isolated config): profile add --driver mysql / --driver mariadb resolve and attempt real connections; inspect fails at connect (no server), not "driver not supported".

Known limitation

The MariaDB driver targets the renamed mariadb-dump/mariadb binaries (MariaDB 10.5+). Older MariaDB installs that ship only mysqldump/mysql are not yet supported (documented in the README).

Deferred to Phase F

binlog incremental, cross-engine sync where MySQL/MariaDB are source or target, CDC, and wiring the RequireCapability gates into verbs (streaming/parallel) once those features exist.

Summary by CodeRabbit

Release Notes

  • New Features

    • Added MySQL and MariaDB database driver support with full backup, restore, and schema inspection capabilities.
  • Tests

    • Enabled integration tests on Ubuntu runners with database container support.
  • Documentation

    • Updated documentation to reflect MySQL and MariaDB availability alongside PostgreSQL support.
  • Chores

    • Added MySQL/MariaDB dependencies and updated CI workflows for extended test coverage.

nixrajput added 10 commits June 21, 2026 22:56
- Add DSN builder, mysqldump/mysql arg builders, fork detection, and
  binary-parameterized Backup/Restore subprocess runners.
- Mirror the postgres stderr-discard pattern (no StderrPipe race);
  pass password via MYSQL_PWD env.
- Add go-sql-driver/mysql; underscore dir stays out of go build ./....
…equirement

go test ./... skips underscore-prefixed dirs, so internal/driver/_mysqlcommon
unit tests were silently excluded from make test and CI. Name _*-prefixed
packages explicitly so the arg-builder/version tests actually run. Also document
that the MariaDB driver requires the renamed mariadb-dump/mariadb binaries.
@coderabbitai

coderabbitai Bot commented Jun 21, 2026

Copy link
Copy Markdown

Review Change Stack

Warning

Review limit reached

@nixrajput, we couldn't start this review because you've reached your PR review rate limit.

More reviews will be available in 24 minutes and 37 seconds. Learn how PR review limits work.

Your organization has run out of usage credits. Purchase more credits in the billing tab to continue.

⌛ How to resolve this issue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based credits.

🚦 How do rate limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan refill rate.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, the refill rate gradually slows as usage increases. The highest same-day bursts are limited more strictly.

Please see our Fair Usage Limits Policy for further information.

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 4830e163-d42c-4c11-a468-10febcaa2194

📥 Commits

Reviewing files that changed from the base of the PR and between 8caf1f4 and f68e95e.

📒 Files selected for processing (8)
  • .github/workflows/ci.yml
  • Makefile
  • README.md
  • internal/driver/_mysqlcommon/args.go
  • internal/driver/_mysqlcommon/args_test.go
  • internal/driver/_mysqlcommon/conn.go
  • internal/driver/_mysqlcommon/dsn.go
  • internal/driver/_testing/suite.go
📝 Walkthrough

Walkthrough

Adds full MySQL and MariaDB driver support (Phase E). A shared internal/driver/_mysqlcommon package provides DSN building, TLS mapping, fork detection, dump/restore arg construction, and a Conn implementation. Thin mysql and mariadb wrapper drivers register capabilities and delegate to the shared layer. Both are wired into the app registry and covered by Testcontainers integration tests. CI and Makefile are updated to explicitly include underscore-prefixed packages and run integration tests on Ubuntu.

Changes

MySQL + MariaDB Driver Implementation

Layer / File(s) Summary
Shared DSN, TLS, fork detection, and dump/restore args
internal/driver/_mysqlcommon/dsn.go, internal/driver/_mysqlcommon/version.go, internal/driver/_mysqlcommon/args.go, internal/driver/_mysqlcommon/args_test.go, internal/driver/_mysqlcommon/version_test.go
DSN and tlsParam construct MySQL connection strings from driver.Profile; Fork enum and DetectFork classify the server by querying VERSION(); BuildDumpArgs/BuildRestoreArgs assemble mysqldump/mysql argument vectors with table inclusion/exclusion support. Unit tests cover all four components.
Shared Conn: open, inspect, backup, restore, verify
internal/driver/_mysqlcommon/conn.go
Implements the Conn struct and all driver.Conn methods: NewConn opens and pings with 3-attempt retry; Inspect queries information_schema.tables; BackupWith/RestoreWith exec the dump/client binary with MYSQL_PWD; Verify computes SHA-256; toolErr maps missing binaries to errs.ErrToolMissing.
MySQL and MariaDB thin wrapper drivers
internal/driver/mysql/driver.go, internal/driver/mariadb/driver.go
Each file registers a Driver via init(), declares engine-specific Capabilities(), implements Connect() by delegating to mysqlcommon.NewConn with the appropriate binary names (mysqldump/mysql or mariadb-dump/mariadb), and includes a compile-time interface check.
App driver registration and integration tests
internal/app/drivers.go, go.mod, internal/driver/mysql/integration_test.go, internal/driver/mariadb/integration_test.go
Blank imports in drivers.go register both new drivers at startup. Integration tests provision MySQL 8.0 and MariaDB 11 containers via Testcontainers, seed a widgets table, and verify row count post-restore through the shared drivertesting.RunDriverSuite harness. go.mod adds go-sql-driver/mysql and the new testcontainers modules.
CI, Makefile, and docs
.github/workflows/ci.yml, Makefile, README.md, CHANGELOG.md, docs/DRIVERS.md
CI adds Ubuntu-only steps to install database client tools and run go test -tags=integration including ./internal/driver/_*/. Makefile test/test-verbose/test-integration targets gain the same explicit path. README, CHANGELOG, and driver docs are updated to reflect Phase E completion and revised tool requirements.

Sequence Diagram(s)

sequenceDiagram
  rect rgba(173, 216, 230, 0.5)
    Note over App,Registry: Driver Registration (init)
    App->>Registry: register mysql.Driver
    App->>Registry: register mariadb.Driver
  end

  rect rgba(144, 238, 144, 0.5)
    Note over Caller,mysqldump: Backup Flow
    Caller->>mysql.Driver: Connect(ctx, profile)
    mysql.Driver->>mysqlcommon.NewConn: NewConn(ctx, profile, "mysqldump", "mysql", "mysql.connect")
    mysqlcommon.NewConn->>sql.DB: Open(DSN) + PingContext (3 retries)
    sql.DB-->>mysqlcommon.NewConn: *sql.DB
    mysqlcommon.NewConn-->>Caller: *Conn

    Caller->>Conn: Backup(ctx, opts, writer)
    Conn->>mysqldump: exec with MYSQL_PWD + BuildDumpArgs
    mysqldump-->>Conn: SQL dump stream
    Conn-->>Caller: nil or toolErr
  end

  rect rgba(255, 200, 100, 0.5)
    Note over IntegrationTest,MariaDB Container: Integration Test Flow
    IntegrationTest->>MariaDB Container: startMariaDB (Testcontainers)
    MariaDB Container-->>IntegrationTest: host:port
    IntegrationTest->>drivertesting.RunDriverSuite: RunDriverSuite(driver, profile, fixtures)
    drivertesting.RunDriverSuite->>Conn: Inspect / Backup / Restore
    drivertesting.RunDriverSuite->>MariaDB Container: SELECT COUNT(*) FROM widgets
    MariaDB Container-->>drivertesting.RunDriverSuite: 2 rows
  end
Loading

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Possibly related PRs

  • nixrajput/siphon#1: Established the PostgreSQL driver and the blank-import registration pattern in internal/app/drivers.go that this PR extends for MySQL and MariaDB.
  • nixrajput/siphon#3: Introduced the Phase D driver hardening layer including driver.Capabilities and drivertesting.RunDriverSuite, which the new MySQL/MariaDB integration tests directly consume.

Poem

🐇 Hippity-hop, a new engine joins the den,
MySQL and MariaDB, now ready and then!
A shared _mysqlcommon keeps the code neat,
MYSQL_PWD set so the dump can complete.
With containers spinning and widgets in store,
Two forks, one harness — who could ask for more? 🥕

🚥 Pre-merge checks | ✅ 3 | ❌ 2

❌ Failed checks (1 warning, 1 inconclusive)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 41.67% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
Title check ❓ Inconclusive The title "Feat/implement phase e" is vague and generic, using a non-descriptive abbreviation without conveying what Phase E actually implements. Use a more specific title that clearly describes the implementation, such as "Implement MySQL and MariaDB drivers with shared _mysqlcommon package" or "Add Phase E: MySQL/MariaDB driver support".
✅ Passed checks (3 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.

✏️ Tip: You can configure your own custom pre-merge checks in the settings.

✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/implement-phase-e

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands and usage tips.

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In @.github/workflows/ci.yml:
- Around line 57-61: Remove the mysql-client package from the apt-get install
command that currently installs postgresql-client, mysql-client, and
mariadb-client. Keep only postgresql-client and mariadb-client, since
mariadb-client provides mysql-compatible binaries and installing both
mysql-client and mariadb-client together can cause nondeterministic CI failures
due to package conflicts on Ubuntu.

In `@internal/driver/_mysqlcommon/args.go`:
- Around line 11-23: The BuildDumpArgs and BuildRestoreArgs functions do not
include the SSLMode from the Profile parameter in the generated CLI arguments,
causing mysqldump and mysql to use default SSL settings instead of the profile's
explicit TLS policy. Add the --ssl-mode flag to the args slice in both functions
by mapping p.SSLMode to the corresponding CLI values (DISABLED, REQUIRED,
VERIFY_CA, VERIFY_IDENTITY, or PREFERRED), ensuring the TLS policy is
consistently applied across all database operations including backup and
restore.

In `@internal/driver/_mysqlcommon/conn.go`:
- Line 129: The issue is that appending MYSQL_PWD to os.Environ() without
removing preexisting MYSQL_PWD entries can result in duplicate environment
variables, causing the tool to read the wrong password value. Fix this by
filtering the environment variables to remove any existing MYSQL_PWD entries
before appending the new one. This filtering needs to be applied at both
locations where cmd.Env is assigned with MYSQL_PWD (at line 129 and line 147),
ensuring only a single, correct MYSQL_PWD entry exists in the command
environment.

In `@internal/driver/_mysqlcommon/dsn.go`:
- Around line 20-23: The DSN function currently uses fmt.Sprintf with manual
string formatting, which fails to properly escape special characters in database
names, usernames, and passwords. Replace the manual string formatting approach
with mysql.Config by constructing a mysql.Config struct with the appropriate
fields from the driver.Profile parameter (User, Password, Host, Port, Database),
setting the ParseTime and TLS parameters appropriately based on p.SSLMode, then
call FormatDSN() on the config object to generate a properly escaped DSN string.

In `@README.md`:
- Line 42: The native-tools sentence on line 42 describes mysqldump and
mariadb-dump support as "in v1.0" (future release), which conflicts with the
current Phase E status indicating these tools already work today. Remove or
modify the "in v1.0" qualifier from the description of mysqldump and
mariadb-dump to accurately reflect that MySQL and MariaDB tooling support is
currently available, not planned for a future version. Update the text to show
these tools are already part of the current functionality rather than a future
feature.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro

Run ID: 9076bb6f-0486-4b2c-b265-a5cb3f2660ed

📥 Commits

Reviewing files that changed from the base of the PR and between 4099cee and 8caf1f4.

⛔ Files ignored due to path filters (1)
  • go.sum is excluded by !**/*.sum
📒 Files selected for processing (17)
  • .github/workflows/ci.yml
  • CHANGELOG.md
  • Makefile
  • README.md
  • docs/DRIVERS.md
  • go.mod
  • internal/app/drivers.go
  • internal/driver/_mysqlcommon/args.go
  • internal/driver/_mysqlcommon/args_test.go
  • internal/driver/_mysqlcommon/conn.go
  • internal/driver/_mysqlcommon/dsn.go
  • internal/driver/_mysqlcommon/version.go
  • internal/driver/_mysqlcommon/version_test.go
  • internal/driver/mariadb/driver.go
  • internal/driver/mariadb/integration_test.go
  • internal/driver/mysql/driver.go
  • internal/driver/mysql/integration_test.go

Comment thread .github/workflows/ci.yml Outdated
Comment thread internal/driver/_mysqlcommon/args.go
Comment thread internal/driver/_mysqlcommon/conn.go Outdated
Comment thread internal/driver/_mysqlcommon/dsn.go
Comment thread README.md Outdated
Two CI failures from PR #4:
- Windows: ./internal/driver/_*/ is a bash glob PowerShell doesn't expand, so
  go test got a literal path and failed. Name _mysqlcommon by import path
  instead (portable across shells); mirror in the Makefile.
- Ubuntu: mariadb-client Conflicts with mysql-client-core, so they can't
  co-install. Install mysql-client from Ubuntu and pull mariadb-client from
  MariaDB's official APT repo, which is built to coexist.
The Cancel_PropagatesToSubprocess subtest raced a 150ms sleep against the
dump duration, then asserted Backup returned non-nil. That's flaky across
engines: mysqldump/mariadb-dump dump the 5 MiB seed far faster than pg_dump,
so the process finished before cancel() fired and Backup returned nil (CI:
TestSuite_MySQL/MariaDB failed, Postgres passed).

Cancel the context BEFORE starting Backup instead. exec.CommandContext then
refuses to start (or immediately kills) the subprocess, so Backup returns a
non-nil error deterministically on every engine — proving ctx is wired to the
process without a timing bet. Drops the now-unneeded seedCancelLoad helper.
…l DSN, dedup MYSQL_PWD

Addresses CodeRabbit review on PR #4:
- BuildDumpArgs/BuildRestoreArgs now pass --ssl-mode (mapped from Profile.SSLMode)
  so backup/restore honor the same TLS policy the connect DSN does, instead of
  falling back to the client default (PREFERRED). Adds tests.
- DSN now uses mysql.Config.FormatDSN (the driver's canonical builder) instead of
  hand-formatting. Note: FormatDSN escapes the DBName path but not user/passwd, so
  the ':'-in-username limitation stands (documented honestly).
- withMySQLPwd strips any pre-existing MYSQL_PWD from the child env before setting
  ours, guaranteeing a single, correct value.
- README: drop the stale 'in v1.0' qualifier — MySQL/MariaDB tools work today.

Skipped the CI apt-package finding: it targets an outdated diff (the mysql-client
+ mariadb-client install line it flags no longer exists; CI now uses mysql-client
from Ubuntu + mariadb-client from MariaDB's repo, the native-client design).
…orks)

The --ssl-mode flag added for CodeRabbit's SSLMode-propagation finding broke CI:
mariadb-dump rejects --ssl-mode (MariaDB uses --ssl/--skip-ssl, not MySQL's
flag), so both BackupRestore_Roundtrip suites failed with exit 7. SSLMode is
still honored by the connect DSN; propagating it to the dump/restore tools needs
per-fork flag handling and is tracked as a follow-up (see the NOTE in args.go).
Keeps the other review fixes (canonical DSN, MYSQL_PWD dedup, README).
@nixrajput nixrajput merged commit 4135acc into main Jun 21, 2026
5 checks passed
@nixrajput nixrajput deleted the feat/implement-phase-e branch June 21, 2026 19:03
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant